Papers with teacher components
DistillCSE: Distilled Contrastive Learning for Sentence Embeddings (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to sentence embeddings are based on contrastive learning (CL) . |
| Approach: | They propose a framework which performs contrastive learning under the self-training paradigm with knowledge distillation and propose 'Group-P shuffling strategy' and averaging logits from multiple teacher components. |
| Outcome: | The proposed framework outperforms many strong baseline methods and yields a new state-of-the-art performance. |